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Description 

METHOD AND APPARATUS FOR VIDEO 
DECODING AND DE-INTERLACING 

Background of Invention 
[000 1 ] 1. Field of the Invention 

[0002] The invention relates to a method and an apparatus for 
video decoding and de-interlacing, and more particularly, 
to a method and an apparatus for video decoding and de- 
interlacing utilizing a shared storage device. 

[0003] 2. Description of the Prior Art 

[0004] MPEG2 is a standard that has been proposed for the digi- 
tal encoding of video signals. MPEC2 allows for signals to 
be encoded on either an interlaced or progressive basis. 

[0005] The term interlaced is used to refer to image data that is 
represented using, e.g., two alternating image fields. One 
field of an interlaced video frame normally corresponds to 
the odd lines of pixels in the frame with the other field 
corresponding to the even lines of the frame. During dis- 



play, the lines of one field are scanned, e.g., output to a 
display device. The lines of the second field are then 
scanned, so that they are interlaced on the display device 
with the lines of the first field. In this manner, when inter- 
laced images are displayed, odd and even lines of image 
data are updated on an alternating basis. 
[0006] | n the case of progressive display, image data is displayed 
sequentially, e.g., starting at the top left corner of an im- 
age proceeding to the bottom right corner. Thus, in the 
progressive display case, the lines of an image are dis- 
played or updated on a sequential basis without lines be- 
ing skipped. 

[0007] | n a series of progressive video display, the positioning of 
the horizontal display lines is consistent from image to 
image. Accordingly, each newly displayed progressive im- 
age normally will completely replace the previous image 
on a display assuming that the images are the same size. 
In the case of interlaced video display, each frame in- 
cludes two fields which correspond to spatially different, 
e.g., horizontal odd and even lines, of a display device. 
Accordingly, in the case of interlaced images, each field 
updates only a portion of the displayed image. Because 
fields of an interlaced image normally correspond to im- 



ages at different times, merely combining fields 1 and 2 
of an interlaced frame can cause blurring and other image 
distortions when motion is present. For this reason, con- 
version of interlaced images to progressive images nor- 
mally involves some form of motion detection and the ap- 
plication of processing which is a function of detected 
motion. 

[0008] Presently video images are usually encoded to be dis- 
played as interlaced video. Unfortunately, most recent 
display devices such as monitors for computers and pro- 
jectors are designed to display progressive display im- 
ages. 

[0009] The ability to efficiently convert between interlaced and 
progressive image formats continues to increase in im- 
portance due, in part, to the ever increasing use of com- 
puters. Notably, when television scenes or other data rep- 
resented as interlaced images are to be displayed on a 
progressive display, they normally first have to be con- 
verted into progressive image data. 

[0010] High speed memory is normally used in video processing 
applications which convert between image formats. This is 
so that real time, or near real time, processing of video 
data can be achieved. While the cost of memory has 



dropped considerably in recent years, memory still re- 
mains a significant cost component of many image and 
video processing systems. This is because a relatively 
large amount of memory is normally required for video 
applications. In consumer applications and other applica- 
tions where cost is a concern, it is desirable to minimize 
the amount of memory required to implement an image 
processing system or device. 
1 ] Please refer to Fig.l. Fig.l shows a block diagram of a 
video display system 100 according to the prior art. The 
video display system 100 comprises an MPEG2 decoder 
110 receiving MPEG2 coded video data streams from a 
video source (not shown), for decoding the video data to 
generate decoded video data. In order to execute the de- 
coding function with the decoder 110 smoothly and effi- 
ciently, a first memory is coupled to the decoder 110 for 
video frame buffering. The video display system 100 then 
comprises an interlaced display unit 125 coupled to the 
first memory 120. The interlaced display unit 125 is capa- 
ble of processing decoded image data stored in the first 
memory 120 for further displaying on an interlaced dis- 
play (not shown). The video display system 100 also com- 
prises an interlaced/progressive converter 130 coupled to 



the interlaced display unit 125 for de-interlacing inter- 
laced image data from the interlaced display unit 125 to 
generate corresponding progressive video data and out- 
putting the converted video data to a progressive display 
(not shown) for further displaying. Again, in order to exe- 
cute the de-interlacing function with the converter 130, a 
second memory 140 is also coupled to the converter 130 
for video frame buffering. 

[0012] For the prior art decoder 110 to perform the decoding 

function towards different types of video frames, such as I 
picture frames, P picture frames (together called reference 
picture frames), and B picture frames, the first memory 
120 usually contains storage space for buffering at least 
three video frames (i.e., six fields). For the interlaced/ 
progressive converter 130 to perform motion detection 
and as a result switch between intra-field and inter-field 
interpolation, the second memory 140 then requires stor- 
age space for buffering two video frames (i.e., four fields). 
As a whole, to properly execute designed functions of the 
prior art video display system 100, the system 100 re- 
quires a storage capacity of at least five video frames. 

[0013] The prior art interlaced/progressive converter 130 usually 
can perform motion-adaptive de-interlacing operations 



with video information incorporating not more than four 
video fields due to the limited storage space of the second 
memory 140. That is, for each presently displaying frame, 
switching between intra-field and inter-field interpolation 
is based on video information of at most four video fields. 
It is not applicable for the prior art video display system 
100 to perform motion-adaptive de-interlacing opera- 
tions based on an even larger number of fields in order to 
acquire a more precisely predicted display result. 

[0014] As can be seen in Fig.l, operations of the prior art video 
display system 100 involve data exchange between the 
first memory 120 and the decoder 110, between the first 
memory 120 and the interlaced display unit 125, and also 
between the second memory 140 and the interlaced/ 
progressive converter 130. Therefore a total memory 
bandwidth requirement of the prior art video display sys- 
tem 100 is considerably large. This is conventionally not 
desirable for video display system design. 

[0015] | n order to reduce the cost as well as memory bandwidth 
requirement of video systems that perform decoding and 
conversion operations, there is a need for methods and an 
apparatus which allow for a reduction in the amount of 
memory required to implement such systems. In addition, 



in order to perform more well-referenced motion-adap- 
tive de-interlacing operations, methods and an apparatus 
capable of such functionality is also desired. It is desirable 
that any new methods and apparatus be suitable for im- 
plementation in computer systems as well as televisions 

sets, set top boxes, and other video applications. 
Summary of Invention 

[0016] it is therefore a primary objective of the claimed invention 
to provide a method and an apparatus for video decoding 
and de-interlacing utilizing a shared storage device hav- 
ing four frame buffers, to solve the above-mentioned 
problem. 

[0017] According to the claimed invention, a method for video 
decoding in a video decoding/de-interlacing display ap- 
paratus utilizes a storage device having four frame 
buffers. The method includes the following steps: (a) De- 
coding video data of a next picture, (b) If the decoded 
next picture is a B picture, buffering the decoded video 
data of the next picture into a frame buffer of the storage 
device not stored with a reference picture nor a present 
display picture nor a previous display picture, (c) If step 
(b) is not applicable, buffering the decoded video data of 
the next picture into a frame buffer of the storage device 



stored with the previous display picture. 
[0018] The claimed invention further discloses a method for 

video decoding in a video decoding/de-interlacing display 
apparatus utilizing a storage device having four frame 
buffers. The method includes the following steps: (a) De- 
coding video data of a next picture, (b) If the decoded 
next picture is a reference picture, buffering the decoded 
video data of the next picture into a frame buffer of the 
storage device not stored with the last decoded reference 
picture nor a present display picture nor a previous dis- 
play picture. 

[0019] The claimed invention also discloses an apparatus for 
video decoding and de-interlacing. The apparatus in- 
cludes the following.A video decoder for decoding video 
data to generate decoded video data of a next picture. A 
storage device coupled to the video decoder, the storage 
device having four frame buffers, for buffering the de- 
coded video data of the next picture into one of the four 
frame buffers according to data stored in the frame 
buffers.An interlace/progressive converter coupled to the 
storage device, for de-interlacing data stored in the frame 
buffers to generate corresponding progressive video 
data.A controller coupled to the video decoder and the in- 



terlace/progressive converter, for controlling data access 
of the video decoder and the interlace/progressive con- 
verter to the frame buffers of the storage device. 

[0020] it is an advantage of the present invention method and 
apparatus to utilize a shared storage device having only 
four frame buffers between the video decoder and the in- 
terlace/progressive converter. By performing the present 
invention method, the present invention decoding and de- 
interlacing apparatus is capable of accomplishing ad- 
vanced de-interlacing operations such as motion-adaptive 
de-interlacing or 3-2 pull-down recovery operations with 
a lower hardware requirement than the prior art decoding 
and de-interlacing apparatus. 

[0021] These and other objectives of the claimed invention will 

no doubt become obvious to those of ordinary skill in the 

art after reading the following detailed description of the 

preferred embodiment that is illustrated in the various 

figures and drawings. 
Brief Description of Drawings 

[0022] pig.l is a block diagram of a video display system accord- 
ing to the prior art. 

[0023] pig. 2 is a block diagram of an apparatus for video decod- 
ing and de-interlacing according to the present invention. 



[0024] pig. 3 is a flow chart indicating operations of the apparatus 

according to the present invention. 
[0025] pig. 4 is a table indicating decoded video frame buffering 

strategy according to the present invention. 

[0026] Fig. 5 is another table indicating decoded video frame 

buffering strategy according to the present invention. 
Detailed Description 

[0027] please refer to Fig. 2. Fig. 2 shows a block diagram of an 
apparatus 200 for video decoding and de-interlacing ac- 
cording to the present invention. The apparatus 200 com- 
prises a video decoder 210, a storage device 220, an in- 
terlace/progressive converter 230, and a controller 240. 
The video decoder 210 is for decoding video data to gen- 
erate decoded video data of a next picture.The storage 
device 220 is coupled to the video decoder 210. The stor- 
age device 220 has four frame buffers 222 for buffering 
the decoded video data of the next picture into one of the 
four frame buffers 222 according to data stored in the 
frame buffers 222.The interlace/progressive converter 
230 is coupled to the storage device 220 for de- 
interlacing data stored in the frame buffers 222 to gener- 
ate corresponding progressive video data.The controller 
240 is coupled to the video decoder 210 and the inter- 



lace/progressive converter 230 for controlling data access 
of the video decoder 210 and the interlace/progressive 
converter 230 to the frame buffers 222 of the storage de- 
vice 220. 

[0028] usually the de-interlaced progressive video data gener- 
ated by the converter 230 will be directly sent to a pro- 
gressive display (not shown) for further displaying. How- 
ever in another embodiment of the present invention, the 
de-interlaced progressive video data can also be trans- 
mitted to another dedicated storage device, such as a 
DRAM, for storage. Data stored in such a storage device 
can be later accessed by a progressive display for further 
displaying. 

[0029] please refer to Fig. 3. Fig. 3 shows a flow chart indicating 
operations of the apparatus 200 according to the present 
invention. The present invention method comprises the 
following steps: 

[0030] Step 300:Start. 

[0031] step 3 10:Decoding video data of a next picture inputted 
from a video source with the video decoder 210. 

[0032] step 320:Judging with the controller 240 if the next pic- 
ture decoded in step 310 is a B picture or a reference pic- 
ture (that is, an I picture or a P picture). If the next picture 



is a B picture, perform step 330. If the next picture is a 

reference picture, perform step 350. 
[0033] step 330:Buffering the decoded video data of the next 

picture into a frame buffer 222 of the storage device 220 

not stored with a reference picture nor a present display 

picture nor a previous display picture. 
[0034] step 340:lf the condition in step 330 is not applicable, 

buffering the decoded video data of the next picture into 

a frame buffer 222 of the storage device 220 stored with 

the previous display picture. 
[0035] step 350:Buffering the decoded video data of the next 

picture into a frame buffer 222 of the storage device 220 

not stored with the last decoded reference picture nor the 

present display picture. 
[0036] step 360:End. 

[0037] please note that as has been well know in the art, a pic- 
ture (either a B picture, an I picture, or a P picture) re- 
ferred to in the present invention method can be regarded 
as a video frame data, or a video field data that con- 
tributes to half of a frame data, either a top field or a bot- 
tom field of the frame. For the situation when a next pic- 
ture decoded is a video field, if the next picture decoded 
is a field (either a top or a bottom field) of a video frame 



whereof the other field of the same video frame has not 
yet been decoded, the present invention method shown in 
Fig. 3 can be directly performed, while if the next picture 
decoded is a field (either a top or a bottom field) of a 
video frame whereof the other field of the same video 
frame has already been decoded, the next picture de- 
coded can be buffered into the frame buffer previously 
buffered with the other field of the same video frame. 

[0038] By adopting the method shown in the above paragraph 
and in Fig. 3, the present invention apparatus 200 for 
video decoding and de-interlacing is capable of perform- 
ing MPEG type video decoding and interlaced/progressive 
conversion utilizing only a shared storage device having 
four frame buffers, instead of two separate memories 
dedicated to the decoder and the converter, respectively, 
as shown in the prior art. 

[0039] please refer to Fig. 4 and Fig. 5 that show two tables indi- 
cating decoded video frame buffering strategy according 
to the present invention. 

[0040] pig. 4 lists all possible buffering situations in the frame 

buffers 222 of the storage device 220 when the next pic- 
ture decoded is a reference picture (i.e., an I picture or a P 
picture). When considering the next picture and previous 



three latest decoded pictures, there are eight different 
possible decoding sequencing situations in Fig. 4, which 
are the situations of RRRR, RRBR, RBRR, RBBR, BRRR, BRBR, 
BBRR, and BBBR. Please note that in Fig. 4 an R represents 
a reference picture while a B represents a B picture, and 
the number following an R or a B indicates the order of 
said R or said B in the decoding sequence. Also note that 

the notation of B Bl means a plurality of consecutive B 

pictures decoded directly prior to Bl. 
[0041] t he situation of R1R2R3R4 as an example, the previ- 

ous three latest decoded pictures and the next picture in 
the decoding sequence are respectively Rl, R2, R3, and 
R4. Consequently the present display picture R2, the pre- 
vious display picture Rl, and the last reference picture R3 
are stored in three of the four frame buffers 222 of the 
storage device 220. Therefore, when the next picture R4 
is decoded by the video decoder 210, according to the 
present invention method shown in Fig. 3 (step 350) the 
next picture R4 can be buffered into the remaining one 
frame buffer 222 of the storage device 220, as shown in 
Fig. 4. 

[0042] Another example can be shown by examining the situa- 
tion of B1R2B3R4. Similar to the previous example, the 



previous three latest decoded pictures and the next pic- 
ture in the decoding sequence are respectively Bl, R2, B3, 
and R4. Also, note that a reference picture RO appeared 
before Bl in the decoding sequence is also presented in 
Fig. 4. Consequently the present display picture B3, the 
previous display picture RO, and the last reference picture 
B2 are stored in three of the four frame buffers 222 of the 
storage device 220. Therefore, when the next picture R4 
is decoded by the video decoder 210, according to the 
present invention method shown in Fig. 3 (step 350) the 
next picture R4 can be buffered into the remaining one 
frame buffer 222 of the storage device 220, as shown in 
Fig. 4. 

[0043] All the other situations shown in Fig. 4 can be analyzed 
similar to the above two examples for those who are 
skilled in the art to understand that by performing the 
present invention method in Fig. 3, when the next picture 
decoded is a reference picture, there will always be one 
frame buffer available (i.e., the remaining frame buffer) 
for buffering the next picture. 

[0044] | n contrast to Fig. 4, Fig. 5 lists all possible buffering situa- 
tions in the frame buffers 222 of the storage device 220 
when the next picture decoded is a B picture. When con- 



sidering the next picture and previous three latest de- 
coded pictures, there are eight different possible decod- 
ing sequencing situations in Fig. 5, which are the situa- 
tions of RRRB, RRBB, RBRB, RBBB, BRRB, BRBB, BBRB, and 
BBBB. Similar to the convention in Fig. 4, in Fig. 5 an R rep- 
resents a reference picture while a B represents a B pic- 
ture, and the number following an R or a B indicates the 
order of said R or said B in the decoding sequence. The 

notation of B Bl also means a plurality of consecutive 

B pictures decoded directly prior to Bl (in certain situa- 
tion, also directly prior to Rl). 
[0045] Ta|< e the situation of R1R2R3B4 as an example, the previ- 
ous three latest decoded pictures and the next picture in 
the decoding sequence are respectively Rl, R2, R3, and 
B4. Consequently the present display picture R2, the pre- 
vious display picture Rl, and the last two reference pic- 
tures R3, R2 are stored in three of the four frame buffers 
222 of the storage device 220 (note that R2 is repetitively 
categorized as the present display picture and the last two 
reference picture). Therefore, when the next picture B4 is 
decoded by the video decoder 210, according to the 
present invention method shown in Fig. 3 (step 330) the 
next picture R4 will be buffered into the remaining one 



frame buffer 222 of the storage device 220, as shown in 
Fig. 5. 

[0046] Another example can be shown by examining the situa- 
tion of B1B2B3B4. Similar to the previous example, the 
previous three latest decoded pictures and the next pic- 
ture in the decoding sequence are respectively Bl, B2, B3, 
and B4. Also, note that two reference pictures R-l and R0 
appeared before Bl in the decoding sequence is also pre- 
sented in Fig. 4. Consequently the present display picture 
B3, the previous display picture B2, and the last two refer- 
ence pictures R-l, R0 are stored in the four frame buffers 
222 of the storage device 220. Therefore, when the next 
picture R4 is decoded by the video decoder 210, accord- 
ing to the present invention method shown in Fig. 3 (step 
340) the next picture B4 will be buffered into the frame 
buffer 222 stored with the previous display picture, which 
in this case is the frame buffer 222 stored with B2, as 
shown in Fig. 5. 

[0047] All the other situations shown in Fig. 5 can be analyzed 
similar to the above two examples for those who are 
skilled in the art to understand that by performing the 
present invention method in Fig. 3, when the next picture 
decoded is a B picture, there will always be one frame 



buffer available (i.e., the remaining frame buffer or the 
frame buffer with B2, in accordance with different situa- 
tions) for buffering the next picture. 

[0048] Among all the situations defined in Fig. 4 and Fig. 5, there 
are two situations RBBB and BBBB, where the next picture 
decoded is buffered into the frame buffer stored with the 
previous display picture. In order to preserve information 
for both the present display picture and the previous dis- 
play picture at any given time, particularly in the above- 
mentioned two situations, the present invention apparatus 
200 is capable of buffering a decoded next picture into a 
frame buffer following a display operation performed on 
the video data presently stored in the frame buffer. That 
is, the video data stored in the frame buffer has been dis- 
played before it is substituted by the buffering data of the 
decoded next picture. In other word, the buffering opera- 
tion of the decoded next picture never preceeds the dis- 
playing operation of the presently stored video data. 

[0049] | n another embodiment of the present invention video de- 
coding method as previously exemplified by Fig. 4 and 
Fig. 5, for the situations presented in Fig. 4, where the next 
picture decoded is a reference picture, instead of buffer- 
ing the next decoded picture into the remaining picture 



the next decoded picture can also be buffered into the 
frame buffer 222 presently buffered with the previous 
display picture according to the present invention method. 
In order to accomplish this implementation, An apparatus 
200 having a high-speed decoder 210 or an apparatus 
200 capable of buffering the decoded next picture into a 
frame buffer following a display operation, as described in 
the last paragraph, is required. 

[0050] By utilizing the present invention method depicted above, 
the present invention apparatus 200 is capable of per- 
forming motion adaptive de-interlacing operations incor- 
porating video data of 3 through at most 8 fields stored in 
the frame buffers 222 of the storage device 220, since the 
storage device 220 has four frame buffers 222 (i.e., eight 
fields) stored with video data of consecutive video frames. 

[0051] The present invention apparatus 200 is also capable of 
performing recovery operations to video data from a 
telecine source because information for both the present 
display picture and the previous display picture are pre- 
served in the frame buffers 222 of the storage device 220 
at any given time. A movie film video data being con- 
verted through the 3-2 pull-down operation shows field 
combining phenomenon in two out of five video frames. 



That is, a video frame showing field combining phe- 
nomenon incorporates two video fields extracted from 
two consecutive different film frames. By using informa- 
tion of a present and a previous display picture, video 
data from a telecine source can be detected, recovered, 
and displayed by the present invention apparatus 200. 

[0052] | n contrast to the prior art, the present invention method 
and apparatus utilize a shared storage device having only 
four frame buffers between the video decoder and the in- 
terlace/progressive converter. By performing the present 
invention method, the present invention decoding and de- 
interlacing apparatus is capable of accomplishing ad- 
vanced de-interlacing operations such as motion-adaptive 
de-interlacing or 3-2 pull-down recovery operations with 
a lower hardware requirement than the prior art decoding 
and de-interlacing apparatus. 

[0053] Those skilled in the art will readily observe that numerous 
modifications and alterations of the device may be made 
while retaining the teachings of the invention. Accord- 
ingly, that above disclosure should be construed as lim- 
ited only by the metes and bounds of the appended 
claims. 



